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Progression  of  CMMs®  and  15504  -1 
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Progression  of  CMMs  and  15504  -2 
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What  EXACTLY  is  1 5504? 


CMMI 


ISO/IEC  15504  is  an  international  standard  being 
progressed  by  WG10  -  a  working  group  established  by 
ISO/IEC/JTC1  SC7. 

Prescribes  requirements  for  process  assessment;  is  a 
framework  standard. 

Consists  of  5  documents  (15504-1,  15504-5);  note  that 

all  parts  are  now  published. 

Not  the  same  as  ISO/IEC  TR  15504  -  this  is  a  9-part  set  of 
documents  published  as  ISO  Technical  Reports  in  1998. 
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CMMI 

Key  Premise  Underlying  15504 

Assessments  of  process  capability  will  yield  results 
which  are  comparable  as  long  as: 

•  the  process  assessment  model  satisfies  15504 
model  compatibility  requirements 

•  the  assessment  process  satisfies  15504 
assessment  requirements 

•  the  assessment  contexts  are  similar 
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Evolution  of  15504  Specificity 


CMMI 


1st  generation  (pre-Pilanesburg)  -  practices  were  HARD-WIRED  into 
15504  (as  a  normative  component  -  BPG) 

2nd  generation  (pre-IS)  -  process  definitions  were  HARD-WIRED  into 
15504  (as  a  normative  component  - 15504-2) 

3rd  generation  (IS)  -  compliance  requirements  for  Process  Reference 
Models  (PRMs)  HARD-WIRED  into  15504  (as  a  normative  component  - 
15504-2): 

•  PRMs  will  be  externally  specified 

•  12207  includes  as  an  amendment  a  PRM  encompassing 
TR  15504-2  process  definitions  (and  more) 
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Architecture  for  15504 
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CMMI  and  15504 


CMMI, 


A-spec  requirements  relevant  to  international 
standards: 

“2.3  Reference  Documents 

2.3.1  Applicable  ISO/IEC  documents,  including 
ISO/IEC  12207  and  ISO/IEC  15504.” 


“3.1  Development  User  Requirements 

3.1.6  The  CMMI  Product  Suite  shall  be  consistent 
and  compatible  with  ISO/IEC  15504.” 
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Synergy  With  Selected  International 
Standards  -1 

Identify  areas  where  there  are  opportunities  for  synergy 
between  key  international  standards  and  the  CMMI 
Product  Suite 

Exploit  these  opportunities  by  developing  appropriate 
work  products  and/or  liaising  with  appropriate 
individuals  and  organizations 
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CMMI 

Synergy  With  Selected  International 
Standards  -2 

Two  of  the  key  areas  identified  to  date  are  the  ISO 
9000:2000  family  of  standards  and  the  ISO/IEC  15504 
standard 

Note  that  15504  provides  a  mechanism  for 
establishing  important  relationships  to  other 
important  process-related  international  standards 
such  as  ISO  9001:2000,  12207,  15288 
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Carnegie  Mel  Ion 
Software  Engineering  Institute 

Key  Supplements  Needed 

Identification  of  suitable  process  reference  models, 

Mappings  between  CMMI  model  (s)  and  PRM  (s) 
satisfying  15504  -2  requirements, 

Translation  mechanism 
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Translation  Mechanism 

Purpose:  for  representing  the  results  of  an 

assessment  as  a  set  of  process  attribute  ratings 
for  each  process  selected  from  the  specified 
Process  Reference  Model 

How?: 

1 .  direct  translation  of  Process  Assessment  Model 
ratings  into  a  process  profile 

2.  or  the  conversion  of  the  data  collected  during  the 
assessment  (with  the  possible  inclusion  of 
additional  information)  through  further  judgment 
on  the  part  of  the  assessor. 
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Carnegie  Mellon 
Software  Engineering  Institute 

Status 

A  detailed  mapping  has  been  developed  from  CMMI 
to: 

•  the  Measurement  Framework  in  ISO/IEC  15504;  and 

•  the  Process  Reference  Model  included  in  ISO/IEC 
12207. 

A  possible  approach  has  been  developed  for  verifiable 
conversion  of  results. 
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An  Approach  for  Data  Conversion  - 1 


For  the  Process  Dimension: 

•  Collect  data  and  generate  observations  for  all  Specific 
Practices  in  Process  Areas  within  the  scope  of  the 
assessment. 

•  Using  the  mapping  tables,  assign  the  data  observations 
to  all  Process  Outcomes  and  Attributes  indicated  from 
the  mapping. 

-  Thus,  if  a  Specific  Practice  maps  to  multiple  Process 
Outcomes,  assign  the  data  to  all  of  the  possibilities. 

•  Review  the  assigned  data  and  remove  observations  that 
are  not  relevant. 

•  Judge  whether  the  remaining  data  is  adequate  to  enable 
judgement  of  achievement  of  the  outcomes. 
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An  Approach  for  Data  Conversion  -  2 

Where  adequate  data  is  available,  consolidate  the  data 
to  the  process  level  and  rate  the  performance  of  the 
process  (Process  Attribute  1.1). 

If  the  data  is  not  adequate,  either: 

•  Collect  additional  data;  or 

•  Remove  the  process  from  the  15504  scope  of  the 
appraisal. 
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An  Approach  for  Data  Conversion  -  3 


Repeat  the  exercise  for  the  Generic  Practices  and  Process 
Attributes  2.1  -  5.2. 

Additional  data  will  need  to  be  collected  to  address  gaps  in 
the  mapping 

•  Most  notably  in  Process  Attribute  3.2 

In  some  cases,  evidence  from  Advanced  Practices  may  also 
be  relevant  to  a  particular  Process  Attribute  for  a  specific 
Process  or  set  of  Processes. 
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CMMI 

Tool  Support 

To  achieve  this  form  of  conversion  is  complex  and 
difficult,  without  tool  support. 

A  tool  (the  Appraisal  Assistant)  has  been  developed 
for  managing  evidence  and  ratings  in  CMMI 
appraisals,  with  a  specific  focus  on  conversion  of  data 
to  generate  ISO15504  Process  Profiles. 

The  tool  automatically  generates  lists  of  data  for  a 
Process  Attribute  based  upon  detailed  ratings  tables. 

*  It  permits  de-selection  of  data  judged  to  be  not 
relevant. 

•  It  highlights  areas  where  there  may  be  a  lack  of 
relevant  evidence. 
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The  Appraisal  Assistant 
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Model  Details 
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Pilot  Appraisal 


15504-conformant  SCAMPI  A  appraisal  conducted  at 
Critical  Software,  SA  during  February-March  2006 

•  Maturity  level  3  scope 

•  Process  reference  model  embedded  in  ISO/IEC  12207  (amended) 

•  Mapping  between  CMMI  and  12207  provided  by  Software  Quality 
Institute 

•  Appraisal  team  included  3  SCAMPI  Lead  appraisers  ,  one  ISO 
9001  auditor  and  one  SPICE  assessor 

•  Support  automation  provided  by  Software  Quality  Institute  - 
Appraisal  Assistant 

•  Subset  of  15504  process  profiles  generated  as  well  as  selected 
ISO  9001  outcomes. 
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CIWMIc 

Project  Management  Process 

Maps  to  elements  from  the  following  Process  Areas: 

•  Project  Planning 

•  Project  Monitoring  and  Control 

•  Supplier  Agreement  Management 

•  Requirements  Management 

•  Integrated  Project  Management 

•  Quantitative  Project  Management 

With  minor  linkages  to 

•  Organizational  Process  Performance 

•  Risk  Management 

•  Verification 
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Project  Management  Process 
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PA  2.1  -  Process  Management 
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PA  2.2  -  Work  Product  Management 
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PA  3.1  -  Process 
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PA  3.2  -  Process  Deployment 
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Pilot  Appraisal  Report 

Result  considered  a  successful  demonstration  of 
ability  to  fulfill  15504  requirements  and  generate 
15504  process  profile  relative  to  a  separate  process 
reference  model, 

Demonstrated  criticality  of  adequate  supporting 
automation 


Carnegie  Mellon 

Software  Engineering  Institute 


©  2006  by  Carnegie  Mellon  University 


Page  28 


Carnegie  Mel  Ion 
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Pilot  Observations 

Improvement  opportunities  were  identified  for  support 
automation 

Sample  of  12207  processes  were  rated  and  a  sample 
of  9001  clauses  were  examined 

Observed  consistency  between  CMMI  findings  and 
prior  9001  audit  reports  as  well  as  prior  SPICE 
assessments,  given  the  expected  progress  of  the 
improvement  program  at  Critical. 
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CMMI 

Issues  from  the  Pilot  Appraisal 

Model-related: 

•  Deficiencies  previously  identified  in  the  relationship 
between  CMMI  Generic  Practices  and  the  15504 
Measurement  Framework  had  a  significant  impact 
on  assessment  of  Level  3  Process  Attributes. 

Technology-related: 

•  Significant  issues  in  usability  of  the  “translation” 
functions. 

•  Need  to  incorporate  the  ability  to  capture  additional 
evidence  where  needed. 
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Further  effort 

Additional  pilot  appraisals  are  required  to  identify  any 
additional  issues  and  to  raise  confidence  in  the 
translation  mechanism  as  well  as  the  translation 
requirements  of  15504. 

Ideally  a  combined  CMMI  /  SPICE  assessment  using 
two  independent  teams  should  be  undertaken. 

The  ability  to  perform  “bi-directional”  translation 
would  be  useful  (i.e.  determine  CMMI  capabilities  from 
a  SPICE  assessment)  would  be  useful. 
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Beta-2  Release 


Significant  improvements  to  the  Evidence  Registry  and  to 
Ratings  Generation. 

CMMI  -  ISO  15504  mapping  functions  improved  in  line  with 
findings  from  the  Pilot  Appraisal. 

Available  for  download  from 
http://www.sqi.qu.edu.au/AppraisalAssistant 

The  SQI  is  committed  to  two  further  beta  releases  of  the 
Appraisal  Assistant  in  2007. 
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CIVVIWI 

Beta  Test  Programme 

Initial  release  2005 
Beta  2  release  28  July  2006 
503  registered  beta  testers 
217  downloads  of  Beta  2  version 
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More  Information 

The  Appraisal  Assistant: 

•  http://www.sqi.qu.edu.au/AppraisalAssistant 

Contact  Details: 

•  David  Kitson  dhk@sei.cmu.edu 

•  Loretta  Kitson  kamolik@mindsprinq.com 

•  Terry  Rout  T.Rout@qriffith.edu.au 
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